Reinforcement learning control of constrained dynamic systems with uniformly ultimate boundedness stability guarantee

Reinforcement learning (RL) is promising for complicated stochastic nonlinear control problems. Without using a mathematical model, an optimal controller can be learned from data evaluated by certain performance criteria through trial-and-error. However, the data-based approach notorious not guaranteeing stability, which most fundamental property any system. In this paper, classic Lyapunov’s method explored to analyze uniformly ultimate boundedness stability (UUB) solely based on without model. It further shown how RL with UUB guarantee applied dynamic systems safety constraints. Based theoretical results, both off-policy and on-policy algorithms are proposed respectively. As result, controllers of closed-loop system at convergence during learning. The series robotic continuous tasks comparison existing algorithms, achieve superior in terms maintaining safety. qualitative evaluation our shows impressive resilience even presence external disturbances.